Method for human motion analysis, apparatus for human motion analysis, device and storage medium

ABSTRACT

A method for human motion analysis, an apparatus for human motion analysis, a device, and a storage medium. The method includes: acquiring image information captured by a number of photographing devices, where at least one of the number of photographing devices is disposed above a shelf; performing human tracking according to the image information captured by the plurality of photographing devices, and determining position information in space of at least one human body and identification information of the at least one human body; acquiring, according to the position information in space of a target human body of the at least one human body, a target image captured by the photographing device above a shelf corresponding to the position information; and recognizing an action of the target human body according to the target image and detection data of a non-visual sensor corresponding to the position information.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to Chinese Patent Application No.201810720374.9, filed on Jul. 3, 2018, which is hereby incorporated byreference in its entirety.

FIELD

Embodiments of the present disclosure relate to the field ofcommunication technologies, and in particular, to a method for humanmotion analysis, an apparatus for human motion analysis, a device, and astorage medium.

BACKGROUND

In the retail scenario, a human motion needs to be analyzed, whichspecifically relates to human tracking and motion acquisition andrecognition involving purchasing behaviors.

In the prior art, the human tracking is performed by a multi-channelcamera uniformly arranged, specifically, two-dimensional imageinformation captured by the multi-channel camera is used as a basis ofthe human tracking, and the human tracking is performed in thetwo-dimensional image and the position of the human body is determined.However, the human tracking is in accurate in a crowded retail scenario.In addition, the prior art uses a light curtain or an infrared sensor toacquire and recognize actions related to the purchase behaviors, but theposition of the hand acquired through the light curtain or the infraredsensor is inaccurate, causing that the human motion cannot be accuratelyrecognized.

SUMMARY

Embodiments of the present disclosure provides a method for human motionanalysis, an apparatus for human motion analysis, a device, and astorage medium, so as to improve the accuracy of human tracking and theprecision of human motion recognition.

In a first aspect, an embodiment of the present disclosure provides amethod for human motion analysis, including:

acquiring image information captured by a plurality of photographingdevices, where at least one of the plurality of photographing devices isdisposed above a shelf;

performing human tracking according to the image information captured bythe plurality of photographing devices, and determining positioninformation in space of at least one human body and identificationinformation of the at least one human body;

acquiring, according to the position information in space of a targethuman body of the at least one human body, a target image captured bythe photographing device above a shelf corresponding to the positioninformation; and

recognizing an action of the target human body according to the targetimage and detection data of a non-visual sensor corresponding to theposition information.

In a second aspect, an embodiment of the present disclosure provides anapparatus for human motion analysis, including:

a first acquisition module, configured to acquire image informationcaptured by a plurality of photographing devices, where at least one ofthe plurality of photographing devices is disposed above a shelf;

a determination module, configured to perform human tracking accordingto the image information captured by the plurality of photographingdevices, and determine position information in space of at least onehuman body and identification information of the at least one humanbody;

a second acquisition module, configured to acquire, according to theposition information in space of a target human body of the at least onehuman body, a target image captured by the photographing device above ashelf corresponding to the position information; and

a recognition module, configured to recognize an action of the targethuman body according to the target image and detection data of anon-visual sensor corresponding to the position information.

In a third aspect, an embodiment of the present disclosure provides adevice, including:

a memory;

a processor; and

a computer program;

where the computer program is stored in the memory and configured to beexecuted by the processor to implement the method of the first aspect.

In a fourth aspect, an embodiment of the present disclosure provides acomputer readable storage medium, having a computer program storedthereon, where the computer program, when executed by a processor,implements the method of the first aspect.

In the method for human motion analysis, the apparatus for human motionanalysis, the device, and the storage medium according to theembodiments of the present disclosure, image information captured by aplurality of photographing devices is acquired, and the human trackingis performed according to the image information captured by theplurality of photographing devices, where at least one of the pluralityof photographing devices is disposed above a shelf, and since thephotographing device above the shelf can capture the human body in frontof the shelf more completely, the accuracy of the human tracking isimproved; in addition, human motion recognition is performed bycombining the image information captured by the photographing deviceabove the shelf and the detection data of a non-visual sensorsurrounding the human body, thereby improving the precision of the humanmotion recognition.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an application scenario according to anembodiment of the present disclosure;

FIG. 2 is a flowchart of a method for human motion analysis according toan embodiment of the present disclosure;

FIG. 3 is a flowchart of a method for human motion analysis according toanother embodiment of the present disclosure;

FIG. 4 is a schematic structural diagram of an apparatus for humanmotion analysis according to an embodiment of the present disclosure;and

FIG. 5 is a schematic structural diagram of a device according to anembodiment of the present disclosure.

The embodiments of the present disclosure have been shown explicitly bythe drawings described above, and will be described in more detailbelow. These drawings and descriptions are not intended to limit thescope of the ideas disclosed by the present disclosure in any way, butrather to illustrate the concept disclosed by the present disclosure forthose skilled in the art by reference to the specific embodiments.

DETAILED DESCRIPTION OF EMBODIMENTS

Exemplary embodiments will be described in detail herein, and examplesthereof are illustrated in drawings. When the following descriptionrelates to the drawings, unless otherwise indicated, the same number indifferent drawings represents the same or similar elements.Implementations described in following exemplary embodiments do notrepresent all implementations consistent with the present disclosure.Instead, they are merely examples of apparatuses and methods consistentwith some aspects of the present disclosure as described in detail inthe appended claims.

The method for human motion analysis according to the present disclosurecan be applied to the application scenario shown in FIG. 1. As shown inFIG. 1, the application scenario may specifically be a retail scenario,such as a supermarket and a shopping mall. As shown in FIG. 1, theapplication scenario includes a plurality of photographing devices, suchas a photographing device 11, a photographing device 12, a photographingdevice 13, and a photographing device 14. The photographing device 11,the photographing device 12, and the photographing device 13 may bedisposed at the top of a store, and the photographing device 14 may bedisposed above a shelf 15. In addition, the application scenario furtherincludes a device 16 having data processing and image processingfunctions, and the device 16 may be a local terminal device of thestore, such as a computer, or a remote server. The device 16 can receiveimage information captured by the photographing device 11, thephotographing device 12, the photographing device 13, and thephotographing device 14. In addition, a non-visual sensor may bedisposed on the shelf 15, such as a light curtain sensor 18 disposed ona beam 17 of the shelf 15, an infrared sensor 20 disposed on a layerplate 19 of the shelf 15, and the device 16 may acquire the detectiondata of the non-visual sensor. Optionally, the photographing device andthe non-visual sensor can be wired or wirelessly connected to the device16.

The method for human motion analysis according to the present disclosureaims to solve the above technical problems of the prior art.

The technical solutions of the present disclosure and how the technicalsolutions of the present application solve the above technical problemswill be described in detail below with reference to specificembodiments. The following specific embodiments may be combined witheach other, and the same or similar concepts or processes may not berepeated in some embodiments. The embodiments of the present disclosurewill be described below with reference to drawings.

FIG. 2 is a flowchart of a method for human motion analysis according toan embodiment of the present disclosure. The embodiment of the presentdisclosure provides a method for human motion analysis aiming at theabove technical problem of the prior art, and specific steps of themethod are as follows.

Step 201, acquiring image information captured by a plurality ofphotographing devices, where at least one of the plurality ofphotographing devices is disposed above a shelf.

As shown in FIG. 1, the photographing device 11, the photographingdevice 12, and the photographing device 13 which are disposed on the topof the store and the photographing device 14 disposed above the shelf 15capture image information in real time, and send the captured imageinformation to the device 16. Optionally, wired communication orwireless communication can be performed between each of thephotographing devices and the device 16. In this embodiment, thephotographing device may specifically be an RGB camera or an RGB-Dcamera. It is only a schematic illustration herein and does not limitthe number and specific location of the photographing device. It can beunderstood that in other embodiments, the photographing device 11, thephotographing device 12, and the photographing device 13 may not bedisposed on the top of the store, for example, they may be disposed on awall, a corner, or the like of the store, as long as the photographingcoverage of the plurality of photographing devices may cover the store.In addition, the photographing device 14 may not be disposed above theshelf, for example, it may be disposed on the vertical beam of theshelf, as long as the photographing device 14 can capture the customerin front of the shelf, and the number of the photographing deviceinstalled on each shelf is not limited herein.

Step 202, performing human tracking according to image informationcaptured by the plurality of photographing devices, and determiningposition information in space of at least one human body andidentification information of the at least one human body.

After receiving the image information captured in real time by thephotographing device 11, the photographing device 12, the photographingdevice 13, and the photographing device 14, the device 16 performs thehuman tracking according to the image information captured in real timeby each photographing device, and determines 3D position information inspace of the human body captured by each photographing device,identification information of the human body, and 2D positioninformation of human body in the image information captured by eachphotographing device.

For example, images captured by the photographing device 11, thephotographing device 12, the photographing device 13 and thephotographing device 14 include a human body A, a human body B, and ahuman body C. After performing the human tracking, the device 16determines the 3D position information in space and the identificationinformation of the human body A, the human body B, and the human body C,as well as the 2D positions of the human body A, the human body B, andthe human body C in the image captured by each photographing device,respectively.

Step 203, acquiring, according to position information in space of atarget human body of the at least one human body, a target imagecaptured by the photographing device above the shelf corresponding tothe position information.

It can be understood that the human body A, the human body B and thehuman body C can generate various actions in the store, such as theactions of picking up a commodity and returning the commodity. Thedevice 16 can recognize each human action, taking the human body A as anexample, after determining the 3D position information in space of thehuman body A, the device 16 can further determine the photographingdevice above the shelf corresponding to the 3D position information. Forexample, the device 16 determines, according to the 3D positioninformation in space of the human body A, the photographing devicedisposed above the shelf and closest to the human body, such as thephotographing device 14, and determines the image information capturedby the photographing device 14 from all the image information receivedfrom the device 16. It can be understood that the image informationcaptured by the photographing device 14 includes the human body A. Theimage information captured by the photographing device 14 is recorded asthe target image.

Step 204, recognizing an action of the target human body according tothe target image and detection data of a non-visual sensor correspondingto the position information.

In this embodiment, the device 16 can also determine the non-visualsensor closest to the human body A according to the 3D positioninformation in space of the human body A. Specifically, the device 16recognizes the action of the human body A according to the imageinformation captured by the photographing device 14 and the detectiondata of the non-visual sensor closest to the human body A.

Specifically, the recognizing the action of the target human bodyincludes: recognizing an action of picking up a commodity of the targethuman body; and/or recognizing an action of putting down the commodityof the target human body.

For example, the device 16 recognizes the action of picking up thecommodity and/or the action of putting down the commodity of the humanbody A according to the image information captured by the photographingdevice 14 and the detection data of the non-visual sensor closest to thehuman body A.

In the embodiment of the present disclosure, image information capturedby a plurality of photographing devices is acquired, and the humantracking is performed according to the image information captured by theplurality of photographing devices, where at least one of the pluralityof photographing devices is disposed above a shelf, and since thephotographing device above the shelf can capture the human body in frontof the shelf more completely, the accuracy of the human tracking isimproved; in addition, the human action is recognized by combining theimage information captured by the photographing device above the shelfand the detection data of the non-visual sensor surrounding the humanbody, thereby improving the precision of the human action recognition.

FIG. 3 is a flowchart of a method for human motion analysis according toanother embodiment of the present disclosure. On the basis of the aboveembodiment, the recognizing an action of the target human body accordingto the target image and detection data of a non-visual sensorcorresponding to the position information specifically includesfollowing steps:

step 301, acquiring a key point of the target human body in the targetimage, where the key point of the target human body includes a key pointof a hand of the target human body.

For example, after receiving the target image captured by thephotographing device 14, the device 16 acquires the key point of thehuman body A in the target image adopting a human body key pointalgorithm according to the 2D position information of the human body Ain the target image. It can be understood that the key point of thehuman body A includes the key point of the soma of the human body A andthe key point of the hand of the human body A, and the device 16 canestablish a correlation between the human body A and the hand of thehuman body A according to the key point of the human body A and the keypoint of the hand of the human body A in the target image.

Step 302, recognizing the action of the target human body according tothe key point of the hand of the target human body and the detectiondata of the non-visual sensor corresponding to the position information.

Specifically, the device 16 recognizes the action of the human body Aaccording to the key point of the hand of the human body A and thedetection data of the non-visual sensor closest to the human body A. Itcan be understood that the device 16 can receive the target imagecaptured by the photographing device 14 in real time, and extract thekey point of the hand of the human body A from the target image in realtime, but the device 16 does not recognize the action of the hand of thehuman body A according to the key point of the hand of the human body Ain real time, because the action of the hand of the human body A doesnot occur in real time, for example, the human body A does not pick upthe commodity or put it down in real time; the human body A may observethe commodity carefully before picking it up; and the human body A willobserve the commodity carefully after picking it up and will notimmediately put it down. Therefore, if the action of the hand of thehuman body A, for example, picking up a commodity or putting it down,occurs with low frequency, and the device 16 recognizes the action ofthe hand of the human body A according to the key point of the hand ofthe human body A in real time, the amount of calculation of the device16 will be increased. Therefore, in the present embodiment, the device16 can recognize the action of the human body A in combination with thedetection data of the non-visual sensor closest to the human body A andthe key point of the hand of the human body A.

Optionally, the non-vision sensor includes a light curtain sensor whichis disposed on a beam of the shelf facing the customer and configured todetect the occurrence time of the action; and the recognizing the actionof the target human body according to the key point of the hand of thetarget human body and the detection data of the non-visual sensorcorresponding to the position information includes: recognizing,according to the key point of the hand of the target human body, theaction of the target human body at the occurrence time of the actiondetected by the light curtain sensor.

As shown in FIG. 1, the light curtain sensor 18 is disposed on the beam17 of the shelf 15, and in particular, the light curtain sensor 18 canbe strip-shaped, and the light curtain sensor 18 is disposed on theouter edge of the beam 17 facing the customer. Optionally, a pluralityof light curtain sensors are disposed on the outer edge of the beam 17,and the light curtain sensor 18 is only one of them. For example, theplurality of light curtain sensors are sequentially arranged on theouter edge of the beam 17, where the position information of each lightcurtain sensor with respect to the beam 17 is preset.

For example, when the human body A reaches out to pick up commodity 21or put the commodity 21 back, the hand of the human body A will pass byat least one of the plurality of light curtain sensors, such as thelight curtain sensor 18, at this time the light curtain sensor 18 cansend a sensing signal to the device 16. And the device 16 can determinethe occurrence time of the action of the hand of the human body Aaccording to the time when the light curtain sensor 18 sends the sensingsignal, and can determine the occurrence position of the action of thehand of the human body A according to the position information of thelight curtain sensor 18 with respect to the beam 17. Specifically, thedevice 16 recognizes the action of the hand of the human body A, such aspicking up the commodity, putting it back, or other actions, accordingto the key point and change of the hand of the human body A at theoccurrence time of the action of the hand of the human body A.

Optionally, the non-vision sensor further includes an infrared sensorwhich is disposed on a layer plate of the shelf; the light curtainsensor is further configured to detect an occurrence position of theaction; the recognizing, according to the key point of the hand of thetarget human body, the action of the target human body at the occurrencetime of the action detected by the light curtain sensor includes:acquiring, at the occurrence time of the action detected by the lightcurtain sensor, according to the occurrence position of the actiondetected by the light curtain sensor, a change of infrared radiationintensity of the human body and/or the commodity detected by theinfrared sensor corresponding to the occurrence position of the action;and recognizing, according to the change of infrared radiation intensityof the human body and/or the commodity detected by the infrared sensorcorresponding to the occurrence position of the action and the key pointof the hand of the target human body, the action of the target humanbody.

As shown in FIG. 1, the infrared sensor 20 is disposed on the layerplate 19 of the shelf 15. It can be understood that the plurality ofinfrared sensors may be disposed on the layer plate 19 of the shelf 15,and the infrared sensor 20 is only one of them. The infrared sensor canbe used to sense the infrared radiated by the hand of the human body Aand the commodity in front of the infrared sensor. When the human body Areaches out to get the commodity 21 or puts the commodity 21 back, theintensity of the infrared radiated by the hand of the human body Adetected by the infrared sensor is constantly changing, for example,when the hand of the human body A gradually approaches the commodity 21,the intensity of the infrared radiated by the hand of the human body Adetected by the infrared sensor is continuously increased; and when thehand of the human body A gradually moves away from the commodity 21, theintensity of the infrared radiated by the hand of the human body Adetected by the infrared sensor is continuously weakened. It can beunderstood that the intensity of the infrared radiated by the hand ofthe human body A detected by the infrared sensor closest to the hand ofthe human body A on the layer plate 19 is relatively accurate.

One possible way to determine the infrared sensor on the layer plate 19closest to the hand of the human body A is: when the human body Areaches out to get the commodity 21 or put the commodity 21 back, thehand of the human body A will pass by the at least one of the pluralityof light curtain sensors as described above, for example, the lightcurtain sensor 18, at this time the light curtain sensor 18 can send asensing signal to the device 16. And the device 16 can determine theoccurrence time of the action of the hand of the human body A accordingto the time at which the light curtain sensor 18 sends the sensingsignal, and can determine the occurrence position of the action of thehand of the human body A according to the position information of thelight curtain sensor 18 with respect to the beam 17. At this time, thedevice 16 can, according to the occurrence position of the action of thehand of the human body A, determine the infrared sensor closest to theoccurrence position of the action of the hand of the human body A, orcan determine the infrared sensor that is closest to the light curtainsensor 18, such as the infrared sensor 20. Further, the device 16acquires intensity change of the infrared radiation of the human bodyand/or the commodity detected by the infrared sensor 20, and recognizesthe action of the hand of the human body A, such as picking up thecommodity, putting the commodity back, or other actions, by combiningthe intensity change of the infrared radiation of the human body and/orthe commodity detected by the infrared sensor 20 with the key point andits change of the hand of the human body A in the target image capturedby the photographing device 14.

Optionally, the non-visual sensor further includes a gravity sensorwhich is disposed on the layer plate of the shelf; the method furtherincludes: acquiring, at the occurrence time of the action detected bythe light curtain sensor, according to the occurrence position of theaction detected by the light curtain sensor, a gravity change detectedby a gravity sensor corresponding to the occurrence position of theaction; and the recognizing the action of the human body according tothe intensity change of infrared radiation of the human body and/or thecommodity detected by the infrared sensor corresponding to theoccurrence position of the action and the key point of the hand of thehuman body includes: recognizing the action of the human body accordingto the intensity change of infrared radiation of the human body and/orthe commodity detected by the infrared sensor corresponding to theoccurrence position of the action, the gravity change of detected by thegravity sensor corresponding to the occurrence position of the action,and the key point of the hand of the human body.

As shown in FIG. 1, the gravity sensor can also be disposed on the layerplate 19 of the shelf 15. It can be understood that the detection resultof the gravity sensor is different when the commodities on the layerplate 19 are reduced or increased, that is, when the human body A picksup the commodity from the layer plate 19 or puts the commodity back, thedetection result of the gravity sensor is different. Therefore, when thedevice 16 recognizes the action of the human body A, the detectionresult of the gravity sensor can also be referred to.

Specifically, when the human body A reaches out to get the commodity 21or put the commodity 21 back, the hand of the human body A will pass byat least one of the plurality of light curtain sensors as describedabove, for example, the light curtain sensor 18, at this time the lightcurtain sensor 18 can send a sensing signal to the device 16. And thedevice 16 can determine the occurrence time of the action of the hand ofthe human body A according to the time at which the light curtain sensor18 sends the sensing signal, and can determine the occurrence positionof the action of the hand of the human body A according to the positioninformation of the light curtain sensor 18 with respect to the beam 17.At this time, the device 16 can, according to the occurrence position ofthe action of the hand of the human body A, determine the infraredsensor closest to the occurrence position of the action of the hand ofthe human body A, or can determine the infrared sensor that is closestto the light curtain sensor 18, such as the infrared sensor 20, and candetermine the gravity sensor closest to the occurrence position of theaction of the hand of the human body A, or can determine the infraredsensor closest to the light curtain sensor 18, such as the gravitysensor. Further, the device 16 acquires the intensity change of theinfrared radiation of the human body and/or the commodity detected bythe infrared sensor 20, and the gravity change detected by the gravitysensor. The device 16 recognizes the action of the hand of the humanbody A, such as picking up the commodity, putting it back, or otheractions, in combination with the intensity change of the infraredradiation of the human body and/or the commodity detected by theinfrared sensor 20, the gravity change detected by the gravity sensor,and the key point and its change of the hand of the human body A in thetarget image captured by the photographing device 14.

In addition, after the device 16 determines the action of the hand ofthe human body A, it may also determine which person in the currentstore is performing the action according to the identificationinformation of the human body A determined in the above human trackingprocess.

The embodiment of the invention recognizes the action of the human bodyin combination of the image information captured by the photographingdevice above the shelf and the detection data of the non-visual sensorsurrounding the human body, thereby improving the accuracy of the humanaction recognition. In addition, the occurrence time of the action ofthe human body is determined by the detection data of the non-visualsensor, the action of the human body is recognized at the occurrencetime of the action of the human body, rather than recognizing the actionof the human body in real time, thus the calculation amount of thedevice can be reduced when the occurrence frequency of the action of thehuman body is low, and the resource utilization rate of the device canbe improved.

FIG. 4 is a schematic structural diagram of an apparatus for humanmotion analysis according to an embodiment of the present disclosure.The apparatus for human motion analysis may specifically be the device16 of the above embodiment, or a component of the device 16. Theapparatus for human motion analysis according to the embodiment of thepresent disclosure can perform the processing flow according to theembodiment of the method for human motion analysis. As shown in FIG. 4,the apparatus for human motion analysis 40 includes: a first acquisitionmodule 41, a determination module 42, a second acquisition module 43,and a recognition module 44, where the first acquisition module 41 isconfigured to acquire image information captured by a plurality ofphotographing devices, and at least one of the plurality ofphotographing devices is disposed above the shelf; the determinationmodule 42 is configured to perform human tracking according to the imageinformation captured by the plurality of photographing devices anddetermine the position information in space of at least one human bodyand identification information of the at least one human body; thesecond acquisition module 43 is configured to acquire a target imagecaptured by the photographing device above the shelf corresponding tothe position information according to the position information in spaceof a target human body in the at least one human body; and therecognition module 44 is configured to recognize the action of thetarget human body according to the target image and the detection dataof the non-visual sensor corresponding to the position information.

Optionally, the recognition module 44 includes an acquisition unit 441and an recognition unit 442; the acquisition unit 441 is configured toacquire a key point of the target human body in the target image, wherethe key point of the target human body includes a key point of a hand ofthe target human body; and the recognition unit 442 is configured torecognize the action of the target human body according to the key pointof the hand of the target human body and the detection data of thenon-visual sensor corresponding to the position information.

Optionally, the non-visual sensor comprises a light curtain sensor, thelight curtain sensor is disposed on the beam of the shelf facing thecustomer, and the light curtain sensor is configured to detect anoccurrence time of the action; and the recognition unit 442 isspecifically configured to: recognize the action of the target humanbody at the occurrence time of the action detected by the light curtainsensor according to the key point of the hand of the target human body.

Optionally, the non-vision sensor further includes an infrared sensorwhich is disposed on a layer plate of the shelf; the light curtainsensor is further configured to detect an occurrence position of theaction; and the acquisition unit 441 is further configured to: acquire,at the occurrence time of the action detected by the light curtainsensor, according to the occurrence position of the action detected bythe light curtain sensor, an intensity change of the infrared radiationof the human body and/or the commodity detected by the infrared sensorcorresponding to the occurrence position of the action; and therecognition unit 442 is specifically configured to: recognize, accordingto the intensity change of the infrared radiation of the human bodyand/or the commodity detected by the infrared sensor corresponding tothe occurrence position of the action and the key point of the hand ofthe target human body, the action of the target human body.

Optionally, the non-vision sensor further includes a gravity sensorwhich is disposed on the layer plate of the shelf; the acquisition unit441 is further configured to: acquire, according to the occurrenceposition of the action detected by the light curtain sensor, a gravitychange detected by a gravity sensor corresponding to the occurrenceposition of the action at the occurrence time of the action detected bythe light curtain sensor; and the recognition unit 442 is specificallyconfigured to: recognize the action of the target human body accordingto the intensity change of infrared radiation of the human body and/orthe commodity detected by the infrared sensor corresponding to theoccurrence position of the action, the gravity change detected by thegravity sensor corresponding to the occurrence position of the action,and the key point of the hand of the target human body.

Optionally, the recognition module 44 is specifically configured to:recognize an action of picking up the commodity of the target humanbody; and/or recognize an action of putting down the commodity productof the target human body.

The apparatus for human motion analysis of the embodiment shown in FIG.4 can be used to implement the technical solution of the above methodembodiment, and the implementation principle and the technical effectthereof are similar, which will not be repeated herein.

FIG. 5 is a schematic structural diagram of a device according to anembodiment of the present disclosure. The device can be a terminaldevice or a server. The device according to the embodiment of thepresent disclosure may perform the processing flow according to theembodiment of the method for human motion analysis. As shown in FIG. 5,a device 50 includes a memory 51, a processor 52, a computer program,and a communication interface 53, where the computer program is storedin the memory 51, and is configured to be executed by the processor 52perform the method for human motion analysis described in the aboveembodiments.

The device of the embodiment shown in FIG. 5 can be used to perform thetechnical solutions of the above method embodiment, and theimplementation principle and the technical effect thereof are similar,which will not be repeated herein.

In addition, this embodiment further provides a computer readablestorage medium having a computer program stored thereon, where thecomputer program, when executed by the processor, implements the methodfor human motion analysis described in the above embodiments.

In the several embodiments according to the present disclosure, itshould be understood that the disclosed apparatuses and methods may beimplemented in other manners. For example, the apparatus embodimentsdescribed above are merely illustrative. For example, the division ofthe unit is only a logical function division, and there may be otherdivision manners in actual implementation; for example, multiple unitsor components may be combined or may be integrated into another system,or some features can be ignored or not be executed. In addition, themutual coupling or direct coupling or communication connection shown ordiscussed may be an indirect coupling or communication connectionthrough some interfaces, apparatuses or units, and may be in electrical,mechanical or other forms.

The units described as a separate component may or may not be physicallyseparated, and the components displayed as units may or may not bephysical units, that is, may be located in one place, or may bedistributed to multiple network units. Some or all of the units may beselected as required to achieve the purpose of the solutions of theembodiments.

In addition, each functional unit in each embodiment of the presentdisclosure may be integrated into one processing unit, or each unit mayexist physically separately, or two or more units may be integrated intoone unit. The above integrated unit can be implemented in the form ofhardware or in the form of hardware plus software functional units.

The above integrated unit implemented in the form of a softwarefunctional unit can be stored in a computer readable storage medium. Theabove software functional unit stored in a storage medium includes someinstructions for causing a computer device (may be a personal computer,a server, or a network device, etc.) or a processor to perform part ofthe steps of the methods of each embodiment of the present disclosure.The above storage medium includes: a U disk, a mobile hard disk, aread-only memory (ROM), a random access memory (RAM), a magnetic disk,or an optical disk, and the like, which can store program codes.

Those skilled in the art can clearly understand that, for convenienceand brevity of the description, only the division of each functionalmodule described above is illustrated with examples. In practicalapplications, the above functions can be assigned to differentfunctional modules as needed for completion, that is, the internalstructure of the apparatus is divided into different functional modulesto perform all or part of the functions described above. The specificworking process of the apparatus described above may refer to thecorresponding processes in the foregoing method embodiments, which willnot be repeated herein.

Finally, it should be noted that the above embodiments are merelyillustrative of the technical solutions of the present disclosure, andare not to be taken in a limiting sense; although the present disclosurehas been described in detail with reference to the above embodiments,those skilled in the art will understand that they may still modify thetechnical solutions described in the above embodiments, or equivalentlysubstitute some or all of the technical features; and the modificationsor substitutions do not deviate the essence of the correspondingtechnical solutions from the scope of the technical solutions of eachembodiment of the present disclosure.

What is claimed is:
 1. A method for human motion analysis, comprising:acquiring image information captured by a plurality of photographingdevices, wherein at least one of the plurality of photographing devicesis disposed above a shelf; performing human tracking according to theimage information captured by the plurality of photographing devices,and determining position information in space of at least one human bodyand identification information of the at least one human body;acquiring, according to the position information in space of a targethuman body of the at least one human body, a target image captured bythe photographing device above a shelf corresponding to the positioninformation; acquiring a key point of the target human body in thetarget image, wherein the key point of the target human body comprises akey point of a hand of the target human body; and recognizing an actionof the target human body according to the key point of the hand of thetarget human body and detection data of a non-visual sensorcorresponding to the position information; wherein the non-visual sensorcomprises a light curtain sensor which is disposed on a beam of theshelf facing a customer and used to detect an occurrence time of theaction and an occurrence position of the action, and an infrared sensorthat is disposed on a layer plate of the shelf; and the recognizing anaction of the target human body according to the key point of the handof the target human body and detection data of a non-visual sensorcorresponding to the position information comprises: acquiring,according to the occurrence position of the action detected by the lightcurtain sensor, an intensity change of infrared radiation of a commoditydetected by the infrared sensor corresponding to the occurrence positionof the action at the occurrence time of the action detected by the lightcurtain sensor; and recognizing, according to the intensity change ofinfrared radiation of the commodity detected by the infrared sensorcorresponding to the occurrence position of the action and the key pointof the hand of the target human body, the action of the target humanbody.
 2. The method according to claim 1, wherein the recognizing anaction of the target human body comprises: recognizing an action ofpicking up a commodity of the target human body.
 3. An apparatus forhuman motion analysis, comprising: a processor and a non-transitorycomputer-readable medium for storing program codes, which, when executedby the processor, cause the processor to: acquire image informationcaptured by a plurality of photographing devices, wherein at least oneof the plurality of photographing devices is disposed above a shelf;perform human tracking according to the image information captured bythe plurality of photographing devices, and determine positioninformation in space of at least one human body and identificationinformation of the at least one human body; acquire, according to theposition information in space of a target human body of the at least onehuman body, a target image captured by the photographing device above ashelf corresponding to the position information; acquire a key point ofthe target human body in the target image, wherein the key point of thetarget human body comprises a key point of a hand of the target humanbody; and recognize an action of the target human body according to thekey point of the hand of the target human body and detection data of anon-visual sensor corresponding to the position information; wherein thenon-visual sensor comprises a light curtain sensor which is disposed ona beam of the shelf facing a customer and used to detect an occurrencetime of the action and an occurrence position of the action, and aninfrared sensor that is disposed on a layer plate of the shelf; and theprogram codes further cause the processor to: acquire, according to theoccurrence position of the action detected by the light curtain sensor,an intensity change of infrared radiation of a commodity detected by theinfrared sensor corresponding to the occurrence position of the actionat the occurrence time of the action detected by the light curtainsensor; and recognize, according to the intensity change of infraredradiation of the commodity detected by the infrared sensor correspondingto the occurrence position of the action and the key point of the handof the target human body, the action of the target human body.
 4. Theapparatus according to claim 3, wherein the program codes further causethe processor to: recognize an action of picking up a commodity of thetarget human body.
 5. The method of claim 1, wherein the human motionanalysis is performed by a device comprising: a memory; a processor; anda computer program; wherein the computer program is stored in the memoryand is configured to be executed by the processor.
 6. A non-transitorycomputer readable storage medium, having a computer program storedthereon, wherein the computer program, when executed by a processor,implements the method according to claim
 1. 7. The method according toclaim 1, wherein the non-vision sensor further comprises a gravitysensor that is disposed on the layer plate of the shelf; the methodfurther comprises: acquiring, according to the occurrence position ofthe action detected by the light curtain sensor, a gravity changedetected by the gravity sensor corresponding to the occurrence positionof the action at the occurrence time of the action detected by the lightcurtain sensor; and the recognizing, according to the intensity changeof infrared radiation of the commodity detected by the infrared sensorcorresponding to the occurrence position of the action and the key pointof the hand of the target human body, the action of the target humanbody comprises: recognizing, according to the intensity change ofinfrared radiation of the commodity detected by the infrared sensorcorresponding to the occurrence position of the action, the gravitychange detected by the gravity sensor corresponding to the occurrenceposition of the action, and the key point of the hand of the targethuman body, the action of the target human body.
 8. The method accordingto claim 1, wherein the recognizing an action of the target human bodycomprises: recognizing an action of putting down a commodity of thetarget human body.
 9. The apparatus according to claim 3, wherein thenon-vision sensor further comprises a gravity sensor that is disposed onthe layer plate of the shelf; the program codes further cause theprocessor to: acquire, according to the occurrence position of theaction detected by the light curtain sensor, a gravity change detectedby the gravity sensor corresponding to the occurrence position of theaction at the occurrence time of the action detected by the lightcurtain sensor; and recognize, according to the intensity change ofinfrared radiation of the commodity detected by the infrared sensorcorresponding to the occurrence position of the action, the gravitychange detected by the gravity sensor corresponding to the occurrenceposition of the action, and the key point of the hand of the targethuman body, the action of the target human body.
 10. The apparatusaccording to claim 3, wherein the program codes further cause theprocessor to: recognize an action of putting down the commodity of thetarget human body.